-
-
Notifications
You must be signed in to change notification settings - Fork 661
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
UIA objects: introduce a global suggestion list item class for Windows 10. re #6241 #6274
UIA objects: introduce a global suggestion list item class for Windows 10. re #6241 #6274
Conversation
Hi, maybe we should create a UIA behaviors package? Sent from a mobile device.
|
Hi, UIA behaviors: I'm thinking not. |
@michaelDCurran, I'd like to request a review please. Thanks. |
source/NVDAObjects/UIA/__init__.py
Outdated
if self.UIAElement.cachedAutomationID in ("TextBox", "SearchTextBox"): | ||
clsList.append(SearchField) | ||
try: | ||
if self.parent.UIAElement.cachedAutomationId.lower()=="suggestionslist": |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This could be very costly and could end up being recursive -- I had made the same mistake with Microsoft Edge code earlier this year. Items within a suggestionList don't have a useful UIA className or automationID?
source/NVDAObjects/UIA/__init__.py
Outdated
def event_UIA_controllerFor(self): | ||
# Only useful if suggestions appear and disappear. | ||
focus = api.getFocusObject() | ||
if len(self.controllerFor)>0: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you'd want to check that self is focus before checking len(self.controllerFor)... You already fetched focus but didn't use it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When I tested it with Settings and what not, just using self.controllerFor was enough, but I'd be happy to check the focus to really make sure (thanks for catching this).
source/NVDAObjects/UIA/__init__.py
Outdated
if self.UIAElement.cachedAutomationID in ("TextBox", "SearchTextBox"): | ||
clsList.append(SearchField) | ||
try: | ||
if self.parent.UIAElement.cachedAutomationId.lower()=="suggestionslist": |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If it is absolutely necessary to check the parent, then you can use raw UIA calls. Using the UIAHandler.handler.baseTreeWalker and get the parent element for self.UIAElement and check its automationID.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. I'll read EdgeRS1 branch to see what I can learn from there before I proceed. Thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. I'll read EdgeRS1 branch to see what I can learn from there before I proceed. Thanks.
@josephsl: how did you go with removing the call to self.parent when detecting suggestionLists? If it is necessary still, then you should use raw UIA calls to stop possible NVDAObject recursion which could be a large performance hit. |
913a112
to
04a1c0b
Compare
Hi, Behaviors: I agree - after thinking about it, suggestions detection display is something that other API's would like to participate in (not only Start search box in Windows 10, but also address bar in Firefox, Tell Me suggestions in Office 2016 and so on). Search Field in UIA (Windows 10) will now use this behavior. Thanks. |
Hi, Regarding using self.parent: yes, this is the case. I'll try your suggestion of using UIA calls directly. Thanks. |
Note:
UIAHandler.handler.baseTreeWalker is your friend, if you can't use
getCachedParent
…On Tue, Jan 17, 2017 at 10:39 PM, Joseph Lee ***@***.***> wrote:
Hi,
Regarding using self.parent: yes, this is the case. I'll try your
suggestion of using UIA calls directly. Thanks.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#6274 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AFGivd8nLFMSyI5plUb7QAENq_FYSOCoks5rTaWWgaJpZM4JmDqm>
.
--
Derek Riemer: Improving the world one byte at a time!
- University of Colorado Boulder Department of computer science, 4th
year undergraduate student.
- Accessibility enthusiast.
- Proud user of the NVDA screen reader.
- Open source enthusiast.
- Skier.
Personal website <http://derekriemer.com>
|
Attached is a zip file used for announcing appearance/disappearance of auto-suggestions (credit: @derekriemer). Thanks. |
A few thoughts:
|
…s 10. re nvaccess#6241 Windows 10 uses suggestions list for vairous things, including Start menu suggestions, Store recommendations, Settings app and others. Thus introduce NVDAObjects.UIA.SuggestionListItem, which derives its power from searchui.py version (searchui.py version is gone).
Controller For property is used by an element to let another element depend on values given by the source element, useful if suggestions should be listed for a search field among other possibilities. For now, Controller For property has been added to property to event names map in UIA Handler. A UIA object that handles this event (a search box) is next.
This reverts commit a3b1ce5.
Controller For property is used by an element to let another element depend on values given by the source element, useful if suggestions should be listed for a search field among other possibilities. For now, Controller For property has been added to property to event names map in UIA Handler. A UIA object that handles this event (a search box) is next.
… and announce appearance of suggestions. re nvaccess#6241 Coming from Windows 10 App Essentials add-on: a Search Field is now available that'll detect controller for property event and announce either 'suggestions' or 'suggestions closed' if suggestions appear or disappear, respectively. This takes care of both Start Menu and other fields in Windows 10 (works best in Anniversary Update and later).
…roller for object. re nvaccess#6241 Review from Mick Curran (NV Access): Do not waste a function call for fetching focused element. Besides, make sure to check if the focused control is the search field before proceeding to announce appearance of suggestions.
…stions. re nvaccess#6241. Reviewed by Mick Curran (NV Access): announcement of search suggestion is something that NVDA should handle for other API's, such as suggestions in Firefox address bar, search suggestions in universal apps and so on. A new behavior named 'Suggestion' has been added that allows subclasses to provide custom routines when suggestions appear and disappear. This is handled by event_suggestionOpened and event_suggestionClosed, and by default NVDA will speak and braille this event.
…n to announce suggestion appearance. re nvaccess#6241. Reviewed by Mick Curran (NV Access): Based on the new Suggestion behavior mix-in, it is now possible for various objects to provide custom routines to let users know the appearance of suggestions. Thus UIA/Search Field is the first object to use this routine.
…suggestion list (parent). re nvaccess#6241. Reviewed by Mick Curran (NV Access): it is better to use raw UIA for obtaining parent element. However, one must be careful to catch COM and value errors (COM because the element might not be there, and value because NULL pointer access is logged). The raw UIA method was also recommended by Derek Riemer.
…ced in Windows 10 Start menu for consistency with earlier versions of Windows. re nvaccess#6241. iN Windows 10 Start menu, suggestions are announced automaticlaly, so no need to provide suggestion announcements.
…tions' so the behavior can be better described. re nvaccess#6241. If one inherits from 'Suggestion', the overall impression would be that the field is only going to display suggestions nad nothing else. Many suggestions are shown when text is entered, thus it is better to say 'EditableTextWithSuggestions' to better reflect what the behavior actually does. Also, a sound will be played to let users know the appearance of suggestions. (default vlaue).
…configure how auto-suggestions should be announced. re nvaccess#6241
…ss#6241. In the user guide, an explanatory text has been added to describe what auto-suggestions are.
Comments from Mick Curran and Jamie Teh (NV Access): no need for a separate message option when announcing appearance of auto-suggestions, as the sound cue will let the user of this fact. However, braille users should be notified of this regardless of this flag being turned on (deaf-blind users should be notified of this). User guide and settings dialog: changed the control type and label for auto-suggestions setting to reflect change in behavior.
…lement fetcher fails. re nvaccess#6241. In some cases, when Start menu opens, it isn't announced by NVDA. As a result, parent element fetcher fails when trying to instantiate suggestions list item, with a traceback that ends with UnboundLocalError. Catch this by moving the SuggestionsListItem selector to inside of the try block.
…tions close for consistency with other situations (such as browse mode toggle). re nvaccess#6241.
…sh messages. re nvaccess#6241. Suggestion from Davy Kager: provide a way to let braille users read search suggestion items. This is done by emulating some parts of speech.SpeakObjectProperties except the name and position info map will be fetched (position info map fetching is contingent on whether report position info setting is enabled from Object Presentation dialog). Ideally, NVDA objects should have a way to fetch braille flash messages for controls. Also reworded docstring for SuggestionListItem so it cna include other UIA-based suggestion list items such as Windows 8.x search results.
…lle. re nvaccess#6241, nvaccess#6414. Instead of constructing the likely flash message, use a function used as part of Core issue 6414, which is much simpler than constructing the flash message from scratch.
I could swear this was working the other day, but maybe I was running the wrong build. STR:
I think the UIA suggestion list item code needs to be extended to cover items with a parent automation id of "contextMenu". |
Hi, it reports context menu items along with what you’ve typed into the search field, I believe same behavior as in 2017.2. Thanks.
From: James Teh [mailto:notifications@github.com]
Sent: Wednesday, June 14, 2017 4:58 AM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [nvaccess/nvda] UIA objects: introduce a global suggestion list item class for Windows 10. re #6241 (#6274)
I could swear this was working the other day, but maybe I was running the wrong build. STR:
1. Open the Start menu.
2. Search for an app.
3. Press the applications key.
4. Use the up and down arrows to move through the context menu items.
* Expected: NVDA should report items like Pin to Start, Pin to taskbar, etc.
* Actual: It repeatedly reports what you typed into the search box and nothing more.
I think the UIA suggestion list item code needs to be extended to cover items with a parent automation id of "contextMenu".
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#6274 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AHgLkCOpnxu2zT5xEBl056mStLCd0M69ks5sD8rdgaJpZM4JmDqm> .
|
Mick just confirmed this doesn't work for him either. I just figured out
that it works if Windows 10 App Essentials is enabled. If it's disabled, it
doesn't work. However, it does work in master. git bisect also shows that
this PR causes the regression. From the code, I can also see why this
wouldn't work: the suggestion item class only gets used for a specific
parent automation id.
|
Oh great… Just confirmed. An emergency patch is on its way. Thanks.
From: James Teh [mailto:notifications@github.com]
Sent: Wednesday, June 14, 2017 5:42 PM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [nvaccess/nvda] UIA objects: introduce a global suggestion list item class for Windows 10. re #6241 (#6274)
Mick just confirmed this doesn't work for him either. I just figured out
that it works if Windows 10 App Essentials is enabled. If it's disabled, it
doesn't work. However, it does work in master. git bisect also shows that
this PR causes the regression. From the code, I can also see why this
wouldn't work: the suggestion item class only gets used for a specific
parent automation id.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#6274 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AHgLkMnrmiE9qWltJ-t1iyaXH-azrGtYks5sEH3MgaJpZM4JmDqm> .
|
…ontext menu as a suggestions list item. re nvaccess#6241. Oddly, the same behavior that's applied to suggestion items must work in Start suggestion's context menu, otherwise menu items will not be announced.
@josephsl: could you please merge master and resolve the conflict. It is just the addition of the docstring to |
Hi, done. I added docstring there as part of improving the source code documentation. Thanks.
From: Michael Curran [mailto:notifications@github.com]
Sent: Thursday, June 22, 2017 10:35 PM
To: nvaccess/nvda <nvda@noreply.github.com>
Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com>
Subject: Re: [nvaccess/nvda] UIA objects: introduce a global suggestion list item class for Windows 10. re #6241 (#6274)
@josephsl <https://github.com/josephsl> : could you please merge master and resolve the conflict. It is just the addition of the docstring to NVDAObjects.UIA. Then I can merge this to master. Thanks.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#6274 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AHgLkHG9HR4R3vSLmQE4ryGDwky74iP2ks5sG06JgaJpZM4JmDqm> .
|
Windows 10 uses suggestions list for vairous things, including Start menu suggestions, Store recommendations, Settings app and others. Thus introduce NVDAObjects.UIA.SuggestionListItem, which derives its power from searchui.py version (searchui.py version is gone).
Suggested what's new entry:
In Windows 10, you can use up and down arrow keys to review suggestion results (e.g. Settings app suggestions and Store recommendations).
Thanks.